Skip to content

feat(capture): v1 transport + partial-retry send loop (capture v1, 3/6)#703

Draft
eli-r-ph wants to merge 2 commits into
capture-v1/02-serializefrom
capture-v1/03-transport
Draft

feat(capture): v1 transport + partial-retry send loop (capture v1, 3/6)#703
eli-r-ph wants to merge 2 commits into
capture-v1/02-serializefrom
capture-v1/03-transport

Conversation

@eli-r-ph

Copy link
Copy Markdown

💡 Motivation and Context

Third PR in the stacked Capture V1 series (stacked on #702). Adds the HTTP transport and partial-retry send loop for POST /i/v1/analytics/events, on top of the pure transforms from #702. Still inert — send_v1_batch has no caller until the consumer-wiring PR.

New in posthog/capture_v1.py:

  • post_v1(...) — a single attempt. Bearer auth (no api_key in the body), the required v1 headers (PostHog-Sdk-Info, PostHog-Attempt, PostHog-Request-Id, PostHog-Request-Timestamp), and optional gzip. Returns the raw response; this is also the monkeypatch seam the test harness adapter will drive.
  • parse_v1_response(...) — classifies one response without raising: 2xx parses the per-uuid results map (an unparseable 2xx body is flagged malformed), non-2xx best-effort extracts an error message, and Retry-After (delta-seconds or HTTP-date) is parsed in both cases.
  • send_v1_batch(...) — the v1 sibling of Consumer._send. Loops up to max_retries + 1 attempts, but shrinks the batch to only the events the server tagged retry after each 2xx. ok/warning/absent events succeed silently; drop events are logged (a request the server accepted-but-dropped is not a delivery failure, so it is not raised). Raises CaptureV1Error (an APIError subclass, so the consumer's existing on_error(exc, batch) keeps working) on a batch-level terminal/transport failure or once retries are exhausted.
  • CaptureV1Error / V1ParsedResponse / V1EventResult types.

Behavior choices, verified against the Rust contract and posthog-go's capture_v1_send.go:

  • 429 is terminal in v1 (v0 retries it) — the backend signals overload via retryable 5xx + Retry-After.
  • A stable PostHog-Request-Id and created_at span all attempts; PostHog-Attempt increments — so the backend can correlate/dedupe a retried batch.
  • A 2xx with an unparseable body is terminal — re-sending against a broken success would loop forever.
  • Backoff reuses the v0 policy (Retry-After wins, else capped exponential) so both wire protocols back off identically.

Also factors a shared gzip_compress helper out of request.post — pure refactor, no v0 behavior change (covered by the existing test_request.py).

💚 How did you test it?

posthog/test/test_capture_v1.py now has 74 cases (47 transform + 27 transport). New transport coverage: post_v1 header/url/no-key-in-body/gzip-magic; parse_v1_response success/malformed/missing-results/error-body-variants/text-fallback/Retry-After; and send_v1_batch driven by a stubbed post_v1 with mocked sleeps — all-ok, absent-uuid-accepted, partial-retry-shrinks-to-retry-uuids, stable-request-id + incrementing-attempt, drop-logged-not-raised, retry-exhausted-raises, malformed-2xx-terminal, 400/429-terminal-not-retried, 503-then-success (honors Retry-After), 503-exhausted-raises, transport-error-then-success, transport-error-exhausted-reraises.

ruff format/check clean; mypy clean on capture_v1.py + request.py; test_request.py (61) still green after the gzip refactor; regenerated references/public_api_snapshot.txt.

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change (additive; send_v1_batch has no caller yet).

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Authored with Cursor (Claude Opus 4.8) per the agreed plan. posthog-go's sendV1 is woven into its client lifecycle (channels, notifyFailure/notifySuccess, maxAttempts); this port instead fits posthog-python's Consumer._send shape — a synchronous loop that raises on failure so the existing upload() -> on_error(exc, batch) path fires unchanged — while preserving go's partial-retry algorithm, status matrix, stable-request-id semantics, and per-event drop/retry handling.

@greptile-apps

greptile-apps Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "feat(capture): add v1 transport and part..." | Re-trigger Greptile

Comment thread posthog/capture_v1.py Outdated
Comment thread posthog/test/test_capture_v1.py
@github-actions

github-actions Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

posthog-python Compliance Report

Date: 2026-06-28 00:55:20 UTC
Duration: 530116ms

✅ All Tests Passed!

45/45 tests passed


Capture Tests

29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 518ms
Format Validation.Event Has Uuid 10008ms
Format Validation.Event Has Lib Properties 10006ms
Format Validation.Distinct Id Is String 10007ms
Format Validation.Token Is Present 10007ms
Format Validation.Custom Properties Preserved 10007ms
Format Validation.Event Has Timestamp 10007ms
Retry Behavior.Retries On 503 18017ms
Retry Behavior.Does Not Retry On 400 12006ms
Retry Behavior.Does Not Retry On 401 10007ms
Retry Behavior.Respects Retry After Header 16013ms
Retry Behavior.Implements Backoff 30009ms
Retry Behavior.Retries On 500 13018ms
Retry Behavior.Retries On 502 16011ms
Retry Behavior.Retries On 504 16010ms
Retry Behavior.Max Retries Respected 30017ms
Deduplication.Generates Unique Uuids 7002ms
Deduplication.Preserves Uuid On Retry 16015ms
Deduplication.Preserves Uuid And Timestamp On Retry 23015ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 16009ms
Deduplication.No Duplicate Events In Batch 10002ms
Deduplication.Different Events Have Different Uuids 10006ms
Compression.Sends Gzip When Enabled 10007ms
Batch Format.Uses Proper Batch Structure 10007ms
Batch Format.Flush With No Events Sends Nothing 5005ms
Batch Format.Multiple Events Batched Together 10005ms
Error Handling.Does Not Retry On 403 12007ms
Error Handling.Does Not Retry On 413 10007ms
Error Handling.Retries On 408 14014ms

Feature_Flags Tests

16/16 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 9502ms
Request Payload.Flags Request Uses V2 Query Param 10007ms
Request Payload.Flags Request Hits Flags Path Not Decide 10006ms
Request Payload.Flags Request Omits Authorization Header 10007ms
Request Payload.Token In Flags Body Matches Init 10007ms
Request Payload.Groups Round Trip 10006ms
Request Payload.Groups Default To Empty Object 10007ms
Request Payload.Person Properties Distinct Id Auto Populated When Caller Omits It 10007ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 10007ms
Request Payload.Disable Geoip Omitted Defaults To False 10008ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 10006ms
Request Lifecycle.No Flags Request On Init Alone 5003ms
Request Lifecycle.No Flags Request On Normal Capture 10507ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 9511ms
Request Lifecycle.Mock Response Value Is Returned To Caller 10002ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 10510ms

@eli-r-ph eli-r-ph force-pushed the capture-v1/03-transport branch from a901fdc to 7fd7dcd Compare June 27, 2026 23:16
@eli-r-ph eli-r-ph force-pushed the capture-v1/02-serialize branch 2 times, most recently from 419d8ac to 41c6948 Compare June 27, 2026 23:58
@eli-r-ph eli-r-ph force-pushed the capture-v1/03-transport branch from 7fd7dcd to 32d7b02 Compare June 27, 2026 23:58
@eli-r-ph eli-r-ph force-pushed the capture-v1/02-serialize branch from 41c6948 to d6d4aa2 Compare June 28, 2026 00:20
@eli-r-ph eli-r-ph force-pushed the capture-v1/03-transport branch from 32d7b02 to 3677400 Compare June 28, 2026 00:20
@eli-r-ph eli-r-ph self-assigned this Jun 28, 2026
eli-r-ph added 2 commits June 27, 2026 17:38
Adds the HTTP transport for POST /i/v1/analytics/events alongside the
pure transforms: a single Bearer-authed attempt (post_v1) with the
required v1 headers, response classification (parse_v1_response), and the
send loop (send_v1_batch) that resends only the events the server tags
"retry", logs drops, honors Retry-After, and raises CaptureV1Error on
terminal/transport failure or retry exhaustion so the consumer's existing
on_error path fires unchanged.

429 is terminal in v1 (unlike v0). A 2xx with an unparseable body is
terminal to avoid an infinite resend loop. A stable PostHog-Request-Id and
created_at span attempts; PostHog-Attempt increments.

Factors a shared gzip_compress helper out of request.post (no v0 behavior
change). Still inert: nothing calls send_v1_batch until the consumer wiring
PR. 74 capture_v1 tests (47 transform + 27 transport); ruff/mypy clean.
Address review of the v1 transport:
- Hoist the batch created_at out of the retry loop so the envelope stays
  stable across attempts (only the events list and PostHog-Attempt change).
- Isolate v1 request compression behind a CaptureCompression selector
  supporting gzip and zlib-wrapped deflate (RFC 1950), reverting the
  gzip_compress extraction from request.py so the v1 path owns its codecs.
- Stop logging per-event drops at WARNING; a server-chosen drop on a 2xx
  is not a delivery failure and is already carried on CaptureV1Error for
  batch-level surfacing via on_error.
@eli-r-ph eli-r-ph force-pushed the capture-v1/03-transport branch from 3677400 to c698bd5 Compare June 28, 2026 00:45
@eli-r-ph eli-r-ph force-pushed the capture-v1/02-serialize branch from d6d4aa2 to 4ee420a Compare June 28, 2026 00:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant